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Airport  surface  situational  awareness  is  a  flight  crew’s  awareness  of  their  location  with 
respect  to  airport  surface  features  such  as  runways  and  taxiways.  In  conditions  of  low 
visibility,  the  lack  of  airport  surface  situational  awareness  can  lead  an  aircraft  to  enter  an 
active  runway  without  proper  air  traffic  control  (ATC)  clearance.  This  lack  of  a  means  by 
which  pilots  can  safely  navigate  on  the  ground  in  poor  visibility  conditions  has  been  the 
cause  of  many  runway  incursions  and  several  fatal  aircraft  accidents. 

This  study  tested  the  effect  on  airport  surface  situational  awareness  of  GPS-derived 
position  information  depicted  on  a  prototypical  electronic  taxi  chart  display.  The  effects 
of  position  error  and  position  uncertainty  symbology  were  also  tested.  Situational 
awareness  was  assessed  by  asking  12  airline  pilots  a  series  of  questions  about  their 
location  on  the  airport  surface.  The  pilots  used  static  “snapshot”  images  of  a  north-up 
electronic  taxi  chart,  a  supporting  out-the -window  view,  and  an  aircraft  heading  display, 
to  answer  the  situational  awareness  questions. 

Situational  awareness,  as  indicated  by  probe  question  response  accuracy,  increased  when 
aircraft  position  information  was  displayed  on  the  electronic  taxi  chart.  Response  time, 
too,  was  found  to  improve  with  the  presence  of  aircraft  position  information. 
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1.  INTRODUCTION 


The  advent  of  Instrument  Landing  Systems  has  allowed  aircraft  to  safely  takeoff  and  land 
in  low-visibility  conditions.  However,  the  lack  of  a  means  by  which  pilots  can  safely 
navigate  on  the  ground  in  poor  visibility  conditions  has  been  the  cause  of  many  runway 
incursions  and  several  fatal  aircraft  accidents. 

Currently,  flight  crews  use  paper  chart  depictions  of  the  airport  surface  and  out-the- 
window  visual  cues  to  navigate  on  the  surface.  In  addition,  they  can  be  provided  with 
some  feedback  about  their  position  on  the  surface  from  Air  Traffic  Control  (ATC).  In 
clear,  daylight  environmental  conditions  flight  crews  can  correlate  airport  features  and 
navigation  signs  from  the  out-the-window  view  with  the  chart  features  to  maintain  airport 
surface  situational  awareness.  In  conditions  of  fog  and  darkness  however,  out-the- 
window  cues  are  less  available  and  it  becomes  a  difficult  task  for  flight  crews  to  maintain 
situational  awareness.  Low-visibility  conditions  also  prevent  ATC  from  tracking  aircraft 
position  on  the  airport  surface  from  the  tower. 

Airport  surface  situational  awareness  is  a  flight  crew’s  awareness  of  their  location  with 
respect  to  airport  surface  features  such  as  runways  and  taxiways.  In  conditions  of  low 
visibility,  the  lack  of  airport  surface  situational  awareness  may  lead  an  aircraft  to  enter  an 
active  runway  without  proper  ATC  clearance.  This  was  the  case  in  a  ground  collision 
incident  at  Detroit  Metro  Wayne  County  Airport  in  1990.  A  DC-9  mistakenly  entered 
and  proceeded  to  back-taxi  down  the  same  runway  on  which  a  B727  was  cleared  for 
takeoff.  The  727  proceeded  with  the  takeoff  roll  and  a  head-on  collision  resulted.  Due  to 
fog,  tower  controllers  were  not  able  to  see  the  DC-9  taxi  onto  the  active  runway  and 
therefore  were  not  able  to  warn  either  of  the  flight  crews.  This  incident  resulted  in  8 
fatalities  and  21  injuries  [Harrison,  1991]. 

To  provide  some  background  on  the  difficulty  in  maintaining  situational  awareness 
during  low-visibility  taxi  tasks  as  compared  to  other  phases  of  flight,  an  informal  survey 
of  19  airline  pilots  was  conducted.  The  pilots  had  an  average  flight  experience  of  10,250 
flight  hours.  Pilots  were  asked  to  rate  the  difficulty  of  six  phases  of  a  typical  commercial 
flight  in  terms  of  maintaining  situational  awareness  on  a  scale  from  1  to  5.  The  results 
shown  in  Figure  1.1  indicate  that  ground  taxi  was  the  most  difficult  phase  of  flight  to 
conduct  in  low-visibility  conditions,  followed  by  landing  and  takeoff.  The  ground  taxi 
difficulty  rating  was  greater  than  the  difficulty  ratings  of  the  other  phases  of  flight,  at  a 
5%  significance  level  (t  test). 
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Figure  1.1  Plot  of  Difficulty  of  Maintaining  Situational 
Awareness  in  Low-Visibility  Conditions  vs.  Phase  of 
Flight.  l=Not  Difficult  3=Moderately  Difficult  5=Very 
Difficult. 


Currently,  there  are  no  displays  in  commercial  airline  cockpits  which  show  the  aircraft 
location  with  respect  to  local  airport  features  to  help  crews  determine  their  location  on  the 
airport  surface  in  low-visibility  conditions.  However,  the  advent  of  high-precision  global 
positioning  system  (GPS)  navigation  and  display  technology  has  enabled  flight  deck 
electronic  displays  of  the  airport  surface  with  aircraft  position  information.  Aircraft 
position  can  be  determined  using  GPS  to  better  than  100  meters  or  to  even  higher 
accuracy  using  differential  GPS  (DGPS).  Also,  a  study  on  airport  surface  operations 
requirements  performed  by  the  Boeing  Commercial  Airplane  Group  for  NASA  Langley , 
recommended  the  use  of  flight  deck  taxi  displays  with  ownship  position  as  a  component 
of  a  global  solution  to  low-visibility  surface  operation  difficulties  [Groce  et  al.,  1993]. 

The  objectives  of  this  study  were  as  follows: 

•  Determine  the  benefit  of  displaying  aircraft  position  on  a  north-up 
electronic  taxi  chart  in  terms  of  airport  surface  situational  awareness. 

•  Determine  what  effect  position  accuracy  degradation  has  on  pilot 
situational  awareness  using  a  north-up  electronic  taxi  chart.  This 
data  can  be  used  to  determine  position  accuracy  requirements.  Four 
levels  of  position  error  were  tested  ranging  from  4.5  to  90  meters. 

•  Determine  the  benefit  of  graphically  displaying  real-time  knowledge 
of  position  accuracy  as  opposed  to  the  knowledge  of  worst  case 
position  accuracy  of  the  position  sensing  system. 
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In  order  to  measure  the  impact  of  an  electronic  taxi  chart  on  airport  surface  situational 
awareness,  prototypical  electronic  taxi  charts  were  developed  and  a  test  method  was 
developed  which  involved  asking  airline  pilots  a  series  of  situational  awareness  probe 
questions.  The  charts  were  designed  from  a  Jeppesen  Sanderson  airport  surface  chart. 
Federal  Aviation  Administration  (FAA)  standards  for  airport  markings,  and  feedback 
from  airline  pilots.  The  effect  of  the  electronic  taxi  charts  on  situational  awareness  was 
tested  by  asking  1 2  airline  pilots  a  series  of  situational  awareness  probe  questions  in  static 
“snapshot”  scenarios  with  restricted  out-the-window  visibility.  Independent  variables 
were  aircraft  position  error  and  position  uncertainty  symbology.  Dependent  variables 
were  situational  awareness  probe  question  response  accuracy  which  was  a  measure  of 
situational  awareness  and  response  time,  as  well  as  pilot  subjective  measures. 

Chapter  2  of  this  report  provides  background  information  on  runway  incursions,  GPS, 
electronic  taxi  chart  presentation  issues,  paper  airport  surface  charts,  and  low-visibility 
taxi  procedures.  Chapter  3  documents  the  development  of  the  prototypical  electronic  taxi 
chart  format  which  was  used  in  this  study.  Chapters  4  and  5  are  devoted  to  explaining  the 
experimental  method  and  protocol.  A  brief  explanation  of  the  methods  of  data  analyses  is 
offered  in  Chapter  6.  Experimental  results  are  presented  in  Chapter  7.  Finally, 
conclusions  regarding  this  study  are  presented  in  Chapter  8. 
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2. 


BACKGROUND 


This  chapter  will  provide  a  section  on  runway  incursions  and  the  global  positioning 
system  (GPS).  In  addition,  background  will  be  offered  on  electronic  taxi  chart 
presentation  issues,  paper  airport  surface  charts,  and  low-visibility  taxi  procedures. 

2.1  RUNWAY  INCURSIONS 

Runway  incursions  occur  when  an  aircraft,  vehicle,  person,  or  object  gets  in  the  way  of 
an  aircraft  taking  off  or  landing  on  an  active  runway.  The  official  Federal  Aviation 
Administration  (FAA)  definition  is  : 

"  Any  occurrence  at  an  airport  involving  an  aircraft,  vehicle,  person,  or 
object  on  the  ground  that  creates  a  collision  hazard  or  results  in  loss  of 
separation  with  an  aircraft  taking  off,  intending  to  take  off,  landing,  or 
intending  to  land."  [Harrison,  1991] 

Runway  incursions  are  normally  caused  by  human  error,  either  by  the  ATC  controller  or 
the  pilot  or  controller  of  the  surface  vehicle.  When  a  human  error  is  committed  by  the 
pilot  it  is  often  due  to  a  loss  of  airport  surface  situational  awareness. 

Runway  incursions  are  categorized  as  operational  errors,  pilot  deviations,  and 
vehicle/pedestrian  deviations.  Figure  2.1  shows  a  breakdown  of  the  number  of  incursions 
for  each  category  during  the  4-year  period  from  1989  to  1992. 

It  is  not  unusual  for  airline  pilots  to  be  involved  in  a  runway  or  taxiway  incursion.  In 
order  to  provide  some  background  on  runway  incursions,  an  informal  survey  was 
conducted  of  19  active  airline  pilots  with  an  average  flight  experience  of  10,250  hours. 
When  asked  if  they  had  been  involved  in  a  runway  or  taxiway  incursion  or  close  call,  13 
of  the  19  pilots  said  yes. 
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Pilot  Deviations 
Operational  Errors 
Vehicle/Pedestrian  Deviations 


Figure  2.1  Runway  Incursions  Broken  Down  By 
Category  for  the  4- Year  Period  Beginning  in  1989 

[Kasner,  1992]. 


2.2  THE  GLOBAL  POSITIONING  SYSTEM 


One  of  the  key  ingredients  of  the  implementation  of  a  flight  deck  electronic  taxi  chart 
with  ownship  position  is  an  accurate  position  sensing  system.  The  Global  Positioning 
System  (GPS)  is  a  satellite-based  navigation  system  which  transmits  ranging  signals  to 
receivers  which  then  calculate  an  estimate  of  position.  GPS  has  currently  been  certified 
by  the  FAA  for  limited  use  as  a  position  sensor  for  approaches  [Nordwall,  1994]  and  is  a 
likely  candidate  for  use  in  surface  operations.  Issues  that  arise  in  a  discussion  of  GPS  are 
satellite  coverage  and  position  error.  It  is  not  clear  what  value  of  position  error  will  be 
acceptable  for  a  flight  deck  electronic  taxi  chart.  It  is  one  of  the  objectives  of  this 
experiment  to  provide  insight  into  this  issue. 

GPS  position  error  is  defined  as  the  distance  from  the  GPS  predicted  position  to  the 
actual  position.  For  a  position  sensing  system,  an  estimate  of  the  position  error  is 
typically  expressed  as  a  level  of  position  accuracy  or  uncertainty.  This  position 
uncertainty  is  typically  expressed  as  2 o  value  which  means  the  position  error  is  within 
this  range  95%  of  the  time.  For  aircraft  in  flight  a  typical  error  estimate  is  given  in 
vertical  and  horizontal  components.  However,  for  surface  operations  only  a  horizontal 
estimate  of  position  is  required. 
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GPS  position  error  depends  on  two  primary  factors:  the  geometric  configuration  of  the 
satellites  from  which  the  receiver  is  accepting  ranging  signals,  and  the  precision  with 
which  the  GPS  receiver  can  measure  the  ranging  distance  to  each  satellite.  Normally  four 
satellites  are  needed  to  obtain  a  position  fix:  three  to  obtain  latitude,  longitude,  and 
altitude  coordinates  and  one  to  cancel  out  clock  errors  due  to  the  difference  in  time 
between  the  expensive,  precise  clocks  on  the  satellites  and  the  cheaper,  less-precise 
clocks  in  the  GPS  receivers.  However,  for  surface  operations,  only  three  satellites  are 
needed  because  altitude  will  be  known.  Position  error  is  lowest  when  the  satellites  are 
widely  spread  out  with  large  angles  between  them  [Logston,  1992],  The  geometrical 
dilution  of  precision  (GDOP)  is  a  numerical  measure  of  how  well  the  satellites  are 
mutually  positioned. 

GPS  satellites  transmit  on  two  L-band  carrier  frequencies:  LI  and  L2.  The  LI  frequency 
is  modulated  with  the  course  acquisition  (C/A)  code  and  with  the  precise  (P)  code.  The 
L2  carrier  is  modulated  only  with  the  P  code.  The  C/A  code  is  available  to  all  users  while 
the  P  code  is  restricted  to  military  use.  The  Department  of  Defense  (DOD)  intentionally 
degrades  the  C/A  code  ranging  signals  for  civilian  use  by  method  of  Selective 
Availability  (S/A).  The  horizontal  2 a  accuracy  of  GPS  for  civilian  use  is  considered  to 
be  100  meters.  This  level  of  position  accuracy  was  established  as  a  compromise  between 
the  FAA  (Federal  Aviation  Administration)  and  the  DOD  for  civilian  use.  S/A  is  not 
consistently  active;  it  was  turned  off  during  the  Gulf  War  to  allow  coalition  forces  to 
obtain  the  best  GPS  positioning  accuracy  [Logston,  1992],  Currently,  it  is  not  clear 
whether  it  will  remain  on  in  the  future. 

Experimental  tests  have  shown  different  levels  of  position  accuracies.  A  study  was 
completed  in  which  a  ground  vehicle  fitted  with  a  GPS  receiver  was  used  to  determine 
GPS  static  position  accuracy  at  Chicago  O'Hare  International  Airport  in  1992.  The  GPS 
data  was  shown  to  have  a  2 a  accuracy  of  41 .32  meters  for  2,489  trials  [Hoffelt  et  al., 
1992],  It  is  important  to  state  that  these  are  position  accuracy  values  for  the  time  and 
location  stated.  Position  error  will  vary  with  the  number  of  satellites  in  view  which  is 
dependent  on  time  and  location,  as  well  as  the  integrity  of  the  ranging  signal. 

A  method  for  improving  the  position  accuracy  is  differential  GPS  (DGPS)  (Figure  2.2). 
This  method  provides  a  stationary  receiving  station  on  the  ground  at  a  known  location. 
This  differential  station  receives  the  ranging  signals  from  the  satellites  and  calculates  the 
difference  between  the  position  predicted  by  triangulation  and  its  known  position.  This 
correction  factor  can  then  be  transmitted  to  local  aircraft  for  improved  user  position 
accuracy.  DGPS  has  been  shown  to  provide  a  2o  position  accuracy  of  4  to  5  meters 
[Hoffelt  et  al.,  1992].  A  factor  of  DGPS  is  that  it  is  limited  to  use  only  at  airports  or 
regions  which  have  a  differential  receiving  station. 
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Figure  2.2  Schematic  of  Differential  GPS.  For  an 

accurate  position  fix,  three  to  four  satellites  are  required. 

The  typical  output  of  GPS  receivers  is  a  position  fix  consisting  of  latitude,  longitude,  and 
altitude.  In  addition,  some  receivers  will  calculate  horizontal  dilution  of  precision 
(HDOP)  and  vertical  dilution  of  precision  (VDOP),  and  display  an  estimate  of  position 
accuracy. 

GPS  or  DGPS  could  conceivably  be  used  to  provide  position  information  to  display 
aircraft  location  on  an  electronic  taxi  chart.  It  is  also  likely  that  the  position  accuracy 
estimate  could  be  displayed  as  a  measure  of  position  confidence. 

2.3  ELECTRONIC  TAXI  CHART  PRESENTATION  ISSUES 


Electronic  displays  first  appeared  in  aircraft  in  order  to  replace  conventional 
electromechanical  instruments.  The  three  primary  advantages  of  using  an  electronic 
display  is  the  ability  to  systematically  use  color  coding,  the  ability  to  display  a  mixture  of 
pictorial,  text,  and  numeric  formats,  and  the  ability  to  have  the  pilots  call  up  a  variety  of 
formats  on  the  same  piece  of  display  hardware  [Wiener  and  Nagel,  1988].  An  example  of 
an  electronic  display  currently  used  in  glass  cockpit  aircraft  is  the  Electronic  Horizontal 
Situation  Indicator  (EHSI).  The  EHSI  is  a  moving  map  display  used  to  display  navigation 
waypoints  en  route.  An  EHSI  developed  at  the  MIT  Aeronautical  Systems  Laboratory 
(ASL)  based  on  a  757/767  display  is  shown  in  Figure  2.3.  It  is  likely  that  an  electronic 
taxi  display  could  be  utilized  to  provide  navigation  information  and  enhance  pilot  airport 
surface  situational  awareness  using  the  same  display  hardware. 
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Figure  2.3  Electronic  Horizontal  Situation  Indicator 
(EHSI),  Based  On  B757/767  Display.  Actual  Display  Is 
Color. 


An  issue  when  discussing  electronic  maps  is  whether  to  display  the  information  in  a 
north-up  or  track-up  (moving  map)  format.  A  north-up  format  would  display  the  airport 
surface  in  a  north-up  orientation.  A  track-up  format  would  display  the  airport  surface 
with  respect  to  the  ownship  aircraft.  Typically,  the  ownship  aircraft  is  placed  horizontally 
at  the  center  and  vertically  one-third  of  the  way  up  the  chart.  Surrounding  terrain  would 
then  be  displayed.  The  advantages  of  a  track-up  chart  include  the  ability  to  display 
surrounding  terrain  always  with  respect  to  the  aircraft.  This  is  helpful  during  taxi  tasks 
because  the  pilot  does  not  have  to  perform  a  mental  rotation  to  orient  the  map  to  the 
aircraft  heading.  An  advantage  of  a  north-up  format  is  that  there  are  no  text  rotation 
problems  because  the  map  orientation  does  not  change.  For  this  study  a  north-up  taxi 
chart  format  was  developed. 

Several  organizations  have  been  performing  research  in  the  area  of  electronic  taxi  charts. 
NASA  Langley  has  developed  electronic  displays  of  airports  in  Denver  and  Chicago  in 
effort  to  investigate  situational  awareness  and  the  benefit  of  electronic  charts  over 
currently  used  paper  charts  [Hunt,  1993].  The  Harris  Corporation  has  also  developed 
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some  electronic  displays  of  the  airport  surface  in  an  effort  to  find  a  solution  to  the  runway 
incursion  problem  [Kulikowsi  and  Harvey,  1992].  The  Harris  displays  showed  all 
runways  and  taxiways.  In  addition,  displays  of  the  airport  surface  are  being  developed  for 
use  in  the  Airport  Surface  Traffic  Automation  Program  (ASTA).  A  simulated  surface 
radar  display  has  been  developed  and  is  in  use  on  a  demonstration  basis  at  Boston’s 
Logan  International  Airport  [MIT  Lincoln  Laboratory,  1993],  The  display  shows 
runways,  taxiways,  and  ramp  areas  as  well  as  surface  traffic. 

An  issue  that  arises  in  a  discussion  of  displaying  aircraft  position  on  an  electronic  taxi 
chart  is  how  to  display  the  position  accuracy  associated  with  the  position  sensing  system. 
The  worst-case  accuracy  of  the  position  sensing  system  can  be  displayed,  or  alternatively, 
the  real-time  position  uncertainty  can  be  displayed.  A  real-time  display  of  position 
accuracy  would  take  advantage  of  increases  in  position  accuracy  due  to  better  satellite 
coverage  or  other  methods  of  improving  accuracy  such  as  DGPS. 

2.4  PAPER  AIRPORT  SURFACE  CHARTS 

Current  charts  are  plan  view  depictions  of  the  airport  surface  and  surrounding  features. 
They  are  used  by  flight  crews  to  plan  and  navigate  taxi  routes  at  unfamiliar  airports.  Two 
organizations  produce  airport  surface  charts:  the  National  Oceanic  and  Atmospheric 
Administration  (NOAA)  and  Jeppesen  Sanderson,  Inc.  Both  organizations  distribute  the 
airport  surface  charts  in  conjunction  with  Instrument  Approach  Plates  (IAPs).  NOAA 
charts  are  contained  in  bound  booklets  and  redistributed  every  58  days  [Hansman  and 
Mykityshyn,  1990],  Jeppesen  Sanderson  charts  are  contained  in  a  ringed  binder  and  are 
distributed  individually  every  two  weeks. 

An  example  of  a  Jeppesen  Sanderson  airport  surface  chart  is  shown  in  Figure  2.4.  The 
main  portion  of  the  Jeppesen  chart  contains  a  plan  view  schematic  of  every  runway  and 
taxiway  on  the  airfield,  as  well  as  some  features  of  the  surrounding  terrain  such  as 
railroad  tracks  and  objects  of  altitudes  which  may  be  dangerous  to  local  air  traffic.  Most 
of  the  airport  surface  diagrams  are  presented  in  a  north-up  format.  The  top  portion  of  the 
charts  contains  the  name  of  the  airport  and  the  city  in  which  it  is  located  as  well  as 
necessary  radio  frequencies. 
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JEPPESEN  SANDERSON,  INC.,  1987,  1991.  ALL  RIGHTS  RE5ERVED. 


Figure  2.4  Example  of  Jeppesen  Sanderson 
Airport  Surface  Chart.  Reproduced  with  permission 
from  Jeppesen  Sanderson,  Inc. 


2.5  LOW-VISIBILITY  TAXI  PROCEDURES 


Currently  navigation  on  the  airport  surface  is  accomplished  using  the  cockpit  out-the- 
window  view,  a  paper  airport  surface  chart,  and  advice  from  ground  and  ramp  controllers. 
In  low-visibility  conditions,  follow-me  trucks  and  tugs  are  sometimes  used  to  guide  the 
aircraft  to  the  gate  once  it  has  landed.  Flight  crews  use  the  paper  chart  of  the  airport 
surface  to  provide  a  reference  to  the  flight  deck  window  visual  cues.  On  approach  the 
chart  is  typically  retrieved  from  its  binder  within  an  hour  from  touchdown  at  unfamiliar 
airports.  On  departure  it  is  typically  reviewed  at  the  gate. 

Ground  taxi  operations  are  broken  up  into  movement  and  non-movement  areas.  The 
movement  area  covers  all  taxiways  and  runways  and  is  governed  by  ATC  ground  control. 
The  non-movement  area  expands  the  ramp  and  terminal  areas  and  is  governed  by  local 
airline  ramp  controllers  at  more  congested  airports. 

Low-visibility  surface  operations  for  transport  category  aircraft  are  normally  governed  by 
takeoff  and  landing  restrictions.  A  decision  to  takeoff  is  governed  by  Runway  Visual 
Range  (RVR),  which  is  a  measure  of  the  visibility  longitudinally  along  the  runway 
surface  in  feet.  RVR  may  be  measured  at  the  runway  touchdown,  midpoint,  and 
rollout  locations.  Landing  decisions  are  based  on  RVR  and  a  decision  height  at  which  the 
runway  must  be  in  sight.  Takeoff  decisions  are  based  on  RVR.  Approach  and  landing 
RVR  minimums  depend  on  guidance  equipment  at  the  particular  runway  and  on  the 
particular  aircraft.  Typically,  600  feet  RVR  has  been  the  minimum,  although  some 
aircraft  and  runways  are  certified  for  300  feet  RVR. 

When  proper  visibility  conditions  exist  to  permit  takeoffs  and  landings,  ground  taxi 
operations  are  accomplished  with  the  aid  of  lighted  runway  and  taxiway  identification 
signs  and  airport  lighting,  as  well  as  airport  surface  charts  and  communication  with  ATC 
ground  control.  Runways  used  during  very  low-visibility  operations  typically  have  flush- 
mounted  centerline  lights  and  edge  lights,  while  most  taxiways  have  edge  lights.  The 
Surface  Movement  Guidance  and  Control  system  (SMGCS),  outlined  in  an  FAA  advisory 
circular,  calls  for  installation  of  taxiway  centerline  lights  at  airports  conducting  operations 
below  600  feet  RVR  [FAA,  1992]. 
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3.  DEVELOPMENT  OF  A  PROTOTYPICAL  ELECTRONIC  TAXI 

CHART  FORMAT 


In  order  to  test  the  effect  of  an  electronic  taxi  chart  on  airport  surface  situational 
awareness,  it  was  necessary  to  develop  a  prototypical  electronic  taxi  chart  format.  The 
term  electronic  taxi  chart  refers  to  an  electronic  display  of  the  airport  surface  to  be  used 
for  taxiing  purposes. 

3.1  ELECTRONIC  TAXI  CHART 

The  overall  layout  of  the  prototypical  electronic  taxi  chart  format  resembled  that  of  a 
Jeppesen  Sanderson  paper  airport  surface  chart.  One  of  the  prototypical  electronic  taxi 
charts  developed  for  this  study  is  shown  in  Figure  3.1.  The  top  portion  of  the  chart 
contained  radio  frequencies  necessary  for  approach  and  departure  and  the  name  and 
location  of  the  airport.  The  geographical  layout  of  the  airport  lies  in  the  center  and  is  a 
scale  view.  It  included  a  plan  view  presentation  of  the  runways  and  taxiways  with  ID’s 
and  airport  buildings.  In  addition,  the  runway  lengths  in  feet  were  also  displayed. 

Although  the  electronic  chart  resembles  the  Jeppesen  paper  chart,  some  features  not 
present  on  paper  charts  were  incorporated.  For  example,  runway  centerlines,  edgelines, 
and  threshold  markers  were  included  on  the  electronic  charts  as  well  as  taxiway 
centerlines.  The  lengths  and  widths  of  the  runways  and  taxiways,  as  well  as  the  runway 
and  taxiway  markings,  were  depicted  to  scale. 

Color  coding  of  the  electronic  taxi  chart  resembled  the  real  world  to  the  extent  possible. 
Runway,  taxiway,  and  ramp  areas  were  dark  gray  to  be  consistent  with  the  actual 
pavement  color.  Similarly,  runway  centerlines,  edgelines,  and  threshold  markers  were 
white  and  taxiway  centerlines  were  yellow.  The  buildings  were  colored  blue.  A  black 
background  was  used  to  provide  contrast. 
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Figure  3.1  Example  of  Electronic  Taxi  Chart.  Actual  size  shown. 
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Although  the  approach  was  to  have  the  basic  layout  resemble  a  standard  paper  airport 
surface  diagram,  some  modifications  were  made  to  facilitate  using  electronic  media  for 
presentation.  For  example,  the  scale  was  increased  by  a  factor  of  1.13  to  allow  the  airport 
surface  depiction  to  be  as  large  as  possible  but  still  fit  the  constraints  of  the  standard  EFTS 
display  size  (5.625”  by  6.75”).  In  addition,  the  airport  runway  ID  symbology  (Figure  3.2) 
remains  horizontal  regardless  of  the  orientation  of  the  runway  to  avoid  aliasing  effects, 
where  the  runway  ID  symbology  on  Jeppesen  charts  is  oriented  perpendicular  to  the 
respective  runway  centerline. 


r  is  3 

^186°^ 

Figure  3.2  Example  of  Runway  ID  Symbology  on  the 
Electronic  Taxi  Charts.  The  larger  font  is  the  actual 
runway  ID  while  the  smaller  is  the  runway  heading  with 
respect  to  North.  This  symbology  was  modeled  from  the 
runway  ID  symbology  on  Jeppesen  Sanderson  Airport 
Surface  Diagrams.  This  is  the  ID  for  “Runway  18.” 

Taxiway  ID  markings  were  similar  to  the  Jeppesen  paper  chart’s  convention.  The 
taxiways  were  identified  by  an  individual  letter  from  the  English  alphabet  and  presented 
on  the  electronic  chart  in  capital  case.  The  ID  was  placed  as  close  to  the  taxiway  as 
possible  without  obstructing  it. 

Text  on  the  electronic  taxi  chart  was  sized  according  to  Society  of  Automotive  Engineers 
(SAE)  standards.  SAE  recommends  that  electronic  display  letters  and  figures  subtend  not 
less  than  a  minimum  vertical  angle  at  the  design  eye  position  of  the  pilot  who  normally 
uses  the  instruments.  SAE  recommends  a  visual  angle  for  three  types  of  data  [SAE, 
1988]: 

Primary  data  6  milliradians 

Nonessential  and  secondary  data  4  milliradians 

Minor  descriptive  legends  3  milliradians. 

The  runway  ID  symbology  text  as  well  as  the  taxiway  ID  text  and  runway  length  text 
were  considered  to  be  primary  data  for  this  experiment,  and  were  sized  so  that  they  would 
subtend  at  an  angle  not  less  than  6  radians.  A  viewing  distance  of  30  inches  was  used  as 
a  reference  value  for  this  experiment  (Figure  3.3).  The  font  size  used  for  the  aircraft 
heading  in  the  runway  ID  symbology  was  9  point  (this  was  the  smallest  of  the  primary 
data  text).  The  visual  angle  for  the  aircraft  heading  text  was  6.25  milliradians. 
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Viewing  Distance 


Figure  3.3  Schematic  of  Subject  Viewing  Distance  and 
Visual  Angle  Subtended  When  Viewing  Electronic  Taxi 
Chart  Text.  Visual  angle  subtends  height  of  electronic 
taxi  chart  text. 


3.2  AIRCRAFT  POSITION  AND  HEADING  SYMBOLOGY 

The  position  of  the  aircraft  on  the  airport  surface  was  depicted  by  overlaying  ownship 
aircraft  symbology  onto  the  electronic  taxi  chart.  Three  things  were  displayed  with  this 
symbology:  the  predicted  location  of  the  aircraft,  the  uncertainty  of  the  predicted 
location,  and  the  aircraft  heading.  The  predicted  location  was  indicated  by  the  apex  of  a 
triangular  icon.  The  aircraft  cockpit  was  used  as  the  aircraft  reference  location.  The 
position  uncertainty  was  indicated  by  an  uncertainty  circle  centered  at  the  apex  of  the 
triangle  (Figure  3.4).  The  uncertainty  circle  defined  the  disc  within  which  the  cockpit  of 
the  aircraft  was  located.  The  aircraft  heading  was  indicated  by  an  imaginary  bisector  of 
the  base  of  the  triangle  pointing  towards  the  apex.  It  should  be  noted  that  for  this  study 
the  heading  was  assumed  to  be  accurately  known. 
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Figure  3.4  Aircraft  Triangular  Icon  and  Uncertainty 
Circle.  The  uncertainty  circle  defines  the  disc  within 
which  the  cockpit  of  the  aircraft  was  located. 

Two  types  of  uncertainty  circles  were  used,  as  shown  in  Figure  3.5.  The  constant  radius 
uncertainty  circle  indicated  the  worst-case  system  position  accuracy,  while  the  variable 
radius  uncertainty  circle  indicated  the  actual  position  uncertainty.  The  constant  radius 
uncertainty  circle  was  intended  to  provide  the  pilot  with  knowledge  of  the  worst-case 
system  uncertainty,  while  the  variable  radius  uncertainty  circle  was  intended  to  provide 
the  pilot  with  knowledge  of  the  current  position  uncertainty  as  a  measure  of  position 
confidence. 


Variable  Radius  Uncertainty  Circle 


Constant  Radius  Uncertainty  Circle 

Figure  3.5  Ownship  Aircraft  Symbology.  Values  shown 
are  radii  of  the  uncertainty  circles  in  meters.  The  5  m 
uncertainty  circle  collapses  to  a  point. 

The  variable  radius  uncertainty  circle  had  four  different  radii:  5  meters,  25  meters,  50 
meters,  and  100  meters.  These  were  chosen  to  reflect  the  four  different  levels  of  position 
error  used  in  the  study.  The  constant  radius  uncertainty  circle  had  only  one  radius:  100 
meters.  This  value  was  chosen  to  emulate  the  2 a  GPS  position  accuracy  level  of  100 
meters. 
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The  colors  of  the  aircraft  symbology  were  selected  after  prototype  testing  to  be  clearly 
visible  to  the  pilot.  It  was  also  desired  to  provide  contrast  between  the  uncertainty 
symbol  which  represented  aircraft  location  and  the  triangular  icon  which  represented 
aircraft  heading.  Green  was  selected  for  the  triangular  icon  and  yellow  was  selected  for 
the  uncertainty  circle  to  provide  good  contrast  between  each  other  and  the  other 
symbology  on  the  chart. 
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4. 


EXPERIMENTAL  METHOD 


In  order  to  assess  the  effect  of  an  electronic  taxi  chart  with  GPS-derived  aircraft  position 
on  airport  surface  situational  awareness,  an  experimental  method  was  developed.  The 
method  emulated  a  worst-case  scenario  of  total  disorientation  under  low-visibility 
conditions  of  600-feet  Runway  Visual  Range  (RVR),  and  tested  the  ability  of  the 
electronic  taxi  chart  to  reorient  the  subject  pilot.  The  method  provided  the  subject  pilot 
with  static  “snapshot”  views  of  an  electronic  taxi  chart,  as  well  as  a  supporting  out-the- 
window  view  and  aircraft  heading  display.  The  electronic  taxi  chart  depicted  the  airport 
surface  and  sometimes  provided  aircraft  position  information,  while  the  supporting  out- 
the-window  view  and  Electronic  Horizontal  Situation  Indicator  (EHSI)  provided  real- 
world  visual  cues  and  numerical  heading  information,  respectively. 

The  “snapshot”  approach  was  worst  case  in  the  sense  that  the  subject  pilot  did  not  have 
the  history  of  taxiing  to  the  point  on  the  airport  surface  at  which  he  was  asked  the 
situational  awareness  probe  question.  He  was  merely  presented  a  “snapshot”  of  his 
current  situation  with  the  aircraft  in  a  stopped  position. 

Situational  awareness  was  measured  by  asking  the  subject  pilots  a  series  of  forced-choice 
probe  questions  about  their  location  on  the  airport  surface.  The  subjects  were  forced  to 
choose  one  of  two  answer  options.  Because  of  the  forced-choice  nature  of  the  probe 
questions,  the  lowest  expected  response  accuracy  would  be  50%,  which  would  indicate 
simple  guessing  without  any  situational  awareness  being  provided  by  the  displays.  The 
probe  question  method  for  assessing  situational  awareness  was  similar  to  the  one 
discussed  in  Aretz's  The  Design  of  Electronic  Map  Displays,  which  describes  an 
experiment  comparing  a  track-up,  north-up,  and  a  north-up  derivative  display  [Aretz, 
1991]. 

Two  quantities  were  measured:  probe  question  response  accuracy  and  probe  question 
response  time.  Response  accuracy  was  a  measure  of  situational  awareness,  while 
response  time  was  a  measure  of  ease  of  use  of  the  electronic  taxi  chart. 

Each  probe  question  was  asked  with  a  separate  “snapshot”  scenario.  Figure  4.1  is  a  flow 
diagram  explaining  how  the  “snapshot”  scenarios  were  presented  to  the  pilot.  The  first 
situational  awareness  probe  question  was  brought  up  on  the  screen  with  a  keyboard 
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Figure  4.1  Flow  Diagram  of  Presentation  of  Snapshot  Scenarios. 


input.  It  was  then  intended  that  the  pilot  read  the  question  and  answer  choices  prior  to 
viewing  the  “snapshot”  displays  in  order  to  avoid  measuring  the  time  it  took  to  read  and 
understand  the  probe  question.  After  reading  the  questions,  the  subject  then  pressed  the 
middle  mouse  button  to  bring  up  the  situational  awareness  displays.  The  question  was 
then  answered  with  the  left  or  right  mouse  button.  This  action  brought  up  a  new  question, 
This  process  was  repeated  throughout  the  experiment. 

The  remainder  of  this  chapter  is  divided  into  the  Experimental  Facilities  section  and  the 
Experimental  Design  section.  The  former  will  provide  information  on  the  electronic  taxi 
chart,  supporting  simulation,  and  the  automatic  data  collection  system.  The  latter  will 
provide  examples  of  the  situational  awareness  probe  questions  and  describe  the 
simulation  of  the  GPS  position  error  as  well  as  describe  the  experimental  variables,  test 
matrix,  and  counterbalancing. 

4.1  EXPERIMENTAL  FACILITIES 

An  experimental  facility  was  developed  to  allow  the  “snapshot”  evaluations  of  situational 
awareness  with  the  prototypical  electronic  taxi  chart  format  discussed  in  Chapter  3.  The 
facility  consisted  of  the  electronic  taxi  chart,  a  supporting  out-the-window  view  and  the 
EHSI.  The  EHSI  was  used  to  display  aircraft  heading.  A  schematic  of  the  facility  is 
shown  in  Figure  4.2. 

A  Silicon  Graphics  Indigo  workstation  was  used  to  present  the  electronic  taxi  chart  and 
supporting  simulation  mentioned  above.  A  computer  mouse  was  used  by  the 
experimental  subject  to  answer  the  situational  awareness  probe  questions.  The  electronic 
taxi  chart  presented  a  plan  view  of  all  the  runways  and  taxiways  on  the  airport  surface 
with  ID's.  The  out-the-window  view  depicted  the  runways  and  taxiways  from  a 
perspective  viewpoint.  The  EHSI  provided  the  pilot  with  aircraft  heading  information. 

4.1.1  Electronic  Taxi  Chart 

The  format  of  the  electronic  taxi  charts  used  in  this  experiment  was  described  in  Chapter 
3.  Fictitious  airports  were  used  in  the  experiment  to  avoid  prior  knowledge  effects.  Two 
airports  were  charted  based  on  the  geometries  of  Cleveland’s  Hopkins  International 
Airport  and  the  Raleigh  County  Memorial  Airport  in  Beckley,  West  Virginia.  These  two 
geometries  were  rotated  and  flipped  to  make  two  additional  airports  with  similar 
geometry  and  different  orientation.  Four  airports  were  used  in  an  attempt  to  prevent 
pilots  from  becoming  overly  familiar  with  the  airport  layouts  during  the  experiment.  The 
airports  were  selected  to  have  medium  complexity.  Each  airport  had  a  set  of  parallel 
runways  which  were  necessary  for  several  of  the  situational  awareness  probe  questions. 
The  width  of  all  runways  was  150  feet  and  the  width  of  all  taxiways  was  75  feet.  These 
values  were  chosen  to  be  consistent  with  runway  and  taxiway  widths  at  typical  U.S. 
airports. 
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IRIS  INDIGO  DISPLAY 


Figure  4.2  View  of  Experimental  Set-up.  Shown  are  the 
electronic  taxi  chart,  the  out-the-window  view,  the 
electronic  horizontal  situation  indicator,  and  a  display  of 
the  situational  awareness  probe  question. 


4.1.2  Supporting  Simulation 

Supporting  simulation  was  provided  to  emulate  typical  situational  awareness  cues  which 
would  be  available  in  low-visibility  conditions.  Described  below  are  the  out-the-window 
view  and  the  EHSI. 
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Out-the-Window  View 


The  out-the-window  view  from  a  cockpit  altitude  of  15  feet  was  used  to  provide  the 
experimental  subjects  with  real-world  visual  cues.  An  example  of  the  out-the-window 
view  is  shown  in  Figure  4.3.  During  the  experiment,  a  standard  fog  algorithm  was  used 
to  reduce  the  out-the-window  visibility.  For  this  experiment  the  visibility  was  set  to  600 
feet  Runway  Visual  Range  (RVR).  The  value  of  600  RVR  was  chosen  as  a  typical  value 
for  very  low  visibility  surface  operations. 


Figure  4.3  Out-the-Window  View.  Shown  with  fog 
algorithm  depicting  600  feet  RVR.  Size  reduced  by  25%. 

The  RVR  was  calibrated  by  placing  a  50-foot-high  black  square  target  600  feet  from  the 
runway  threshold,  placing  the  aircraft  out-the-window  view  at  the  runway  threshold,  and 
adjusting  the  fog  parameters  so  that  the  square  was  just  visible. 

Electronic  Horizontal  Situation  Indicator 

For  this  experiment  an  Electronic  Horizontal  Situation  Indicator  (EHSI)  was  used  to 
display  aircraft  magnetic  heading  in  a  manner  consistent  with  the  EHSI  in  the  B767.  In 
actual  flight  deck  use  the  EHSI  can  also  be  used  to  display  navigation  waypoints.  The 
EHSI  is  shown  in  Figure  4.4. 
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4.4  Electronic  Horizontal  Situation  Indicator  (EHSI) 

Used  in  Experiment.  Size  reduced  by  25%. 

4.1.3  Automatic  Data  Collection  System 

The  probe  question  response  data  was  automatically  recorded  by  the  experimental 
computer  facility  as  shown  in  Figure  4.5  in  order  to  minimize  experimenter  bias  and  to 
simplify  data  analysis.  The  mouse  buttons  were  used  to  start  and  stop  the  timer  and 
record  the  subject’s  response  to  the  situational  awareness  probe  question.  The  subject’s 
response  was  automatically  compared  to  the  correct  response  in  the  computer  database. 
The  output  of  the  data  collection  system  was  probe  question  response  accuracy  and 
response  time. 
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OUTPUT 


Figure  4.5  Diagram  of  Automatic  Data  Collection 
Process.  This  process  was  repeated  for  all  situational 
awareness  probe  questions. 

4.2  EXPERIMENTAL  DESIGN 

The  experiment  was  designed  to  evaluate  the  effect  of  an  electronic  taxi  chart  with 
position  information  on  airport  surface  situational  awareness.  Initially,  examples  of  the 
situational  awareness  probe  questions  will  be  presented.  Following  will  be  an 
explanation  of  the  GPS  position  error  simulation.  The  experimental  variables,  test 
matrix,  and  counterbalancing  will  then  be  presented. 

4.2.1  Situational  Awareness  Probe  Questions 

The  situational  awareness  probe  questions  were  designed  to  query  the  pilot  about  his  or 
her  position  on  the  airport  surface.  Figures  4.6  and  4.7  are  two  examples  of  the 
situational  awareness  probe  questions  with  the  corresponding  electronic  taxi  chart, 
supporting  out-the-window  view  and  EHSI.  Table  4.1  shows  the  15  situational  awareness 
probe  questions  used  in  the  experiment.  Each  of  the  1 5  probe  questions  was  asked  at  four 
separate  locations  on  the  airport  surface  to  allow  the  use  of  each  question  for  the  four 
position  error  levels  tested.  The  question  asked  at  these  four  locations  will  be  considered 
the  four  versions  of  that  particular  situational  awareness  probe  question. 
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"  You  are  cleared  to  taxi  to  Runway  4R  via  Taxiways  F,  A,  and  B.  Are  you  following  the 
correct  route?  Yes  or 


Heading  =  136° 


c  4L  : 


Figure  4.6  First  Example  of  Situational  Awareness 
Probe  Question  Snapshot  Scenario.  Shown  with  50  m 
uncertainty  circle.  Colors  of  electronic  taxi  chart  inverted 
for  printing  purposes. 
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What  runway  are  you  on?"  4L  or  4R 


Heading  =  40° 


khan 

mmm  metoo 

m7472WB\  07  5  BKW \ \7  7-QnAitpm 


TAXI  CHART 


CTAF  123.0 
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Figure  4.7  Second  Example  of  Situational  Awareness 
Probe  Question  Snapshot  Scenario.  Shown  with  100  m 
uncertainty  circle.  Colors  of  electronic  taxi  chart  inverted 
for  printing  purposes. 
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Situational  Awareness  Probe  Question 

Answer  Options 

1 .  What  runway  are  you  on? 

4L  or  4R 

2.  What  taxiway  are  on? 

A  or  E 

3.  What  runway  are  you  approaching? 

23L  or  23R 

4.  What  taxiway  are  you  approaching? 

E  or  G 

5.  Are  you  on  a  runway  or  a  taxiway? 

Runway  or  Taxiway 

6.  What  taxiway  did  you  just  pass? 

X  or  Y 

7.  What  runway  did  you  just  cross? 

22L  or  22R 

8.  You  are  cleared  to  taxi  to  Runway  9 
via  taxiways  C,  A,  D,  and  E.  Are  you 
following  the  correct  route? 

Yes  or  No 

9.  Runways  9  and  4L  are  active. 

Are  you  on  an  active  runway? 

Yes  or  No 

10.  You  are  cleared  for  takeoff  on  Runway  1R. 
An  aircraft  is  backtaxiing  on  Runway  1L. 
Are  you  on  the  correct  runway? 

Yes  or  No 

1 1 .  An  aircraft  is  approaching  and  will  hold 
short  of  Runway  5R-23L  on  Taxiway  V. 
Can  you  exit  at  the  next  intersection? 

Yes  or  No 

12.  You  have  been  instructed  to  hold  short  of 
runway  5L  due  to  landing  traffic.  Should 
you  take  immediate  action? 

Yes  or  No 

13.  You  are  cleared  for  takeoff  on  runway  4L. 
An  aircraft  is  taxiing  on  Runway  22L. 

Are  you  on  the  correct  runway? 

Yes  or  No 

1 4.  You  are  cleared  into  position  and  hold 
Runway  23R.  Are  you  on  the  correct 
runway? 

Yes  or  No 

15.  Taxiways  A  and  C  are  closed  for 

maintenance.  Are  you  on  a  closed  taxiway?  Yes  or  No 


Table  4.1  The  15  Situational  Awareness  Probe 
Questions  Used  in  the  Experiment. 
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The  situational  awareness  probe  questions  were  designed  as  forced  response  questions. 
The  subjects  were  given  two  answer  choices  and  forced  to  choose  one  of  them.  A  “pass” 
option  was  not  provided.  Therefore  a  response  accuracy  score  of  50%  correct  would 
indicate  the  subject  did  not  perform  any  better  then  if  he  or  she  had  been  guessing. 

Scores  higher  than  50%  indicate  the  experimental  facility  provided  some  increase  in 
situational  awareness 

4.2.2  GPS  Position  Error  Simulation 

The  aircraft  position  and  heading  symbology  overlaid  on  the  electronic  taxi  chart  was 
displaced  a  distance  from  the  actual  location  specified  by  the  position  error  independent 
variable.  The  error  was  simulated  in  a  worst-case  direction  which  was  subjectively 
assessed  by  the  investigator  to  be  the  most  ambiguous.  For  example,  for  the  question, 
What  taxiway  are  you  on?,  the  predicted  location  of  the  aircraft  was  placed  in  a  direction 
towards  another  taxiway  so  it  might  appear  that  the  aircraft  was  actually  on  the  wrong 
taxi  way  (Figure  4.8). 


X 


=  Actual  Aircraft  Position 


=  Placement  of  Ownship  Icon  Which  Made 
the  Question  Most  Difficult  to  Answer 


Figure  4.8  Example  of  Simulation  of  Position  Error  on 
Electronic  Taxi  Chart. 
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4.2.3  Experimental  Variables.  Test  Matrix,  and  Counterbalancing 


Initially,  the  independent  and  dependent  variables  will  be  described.  The  test  matrix  and 
counterbalancing  follow. 

Independent  Variables 

Aircraft  Position  Error  -  There  were  four  levels  of  aircraft  position  error  which  were 
defined  as  the  radial  distance  from  actual  aircraft  location  to  the  predicted  aircraft 
location  on  the  electronic  taxi  chart.  The  four  levels  of  position  error  were  4.5  meters, 
22.5  meters,  45  meters,  and  90  meters.  These  values  were  chosen  to  provide  a  broad 
range  of  values  representing  position  errors  of  differential  GPS  (DGPS)  and  GPS.  These 
values  were  90%  of  the  system  accuracy  guaranteed  by  the  variable  radius  uncertainty 
circle. 

Position  Uncertainty  Symbology  -  There  were  two  levels  of  this  independent  variable.  A 
constant  radius  uncertainty  circle  and  a  variable  radius  uncertainty  circle  (please  refer  to 
Figure  3.5).  The  uncertainty  circle  defines  the  disc  in  which  the  cockpit  of  the  simulated 
aircraft  lies.  The  constant  radius  uncertainty  circle  provided  radius  of  100  meters  for  all 
actual  position  error  values.  This  worst  case  value  was  selected  to  emulate  the  GPS  2 o 
error  of  100  meters  in  the  horizontal  plane.  The  variable  radius  uncertainty  circle 
reflected  the  current  system  position  accuracy.  The  four  sizes  of  the  variable  radius 
uncertainty  circle  were  5  meters,  25  meters,  50  meters,  and  1 00  meters.  This  uncertainty 
circle  provided  the  pilot  knowledge  of  the  actual  system  position  accuracy. 

Dependent  Variables 

Probe  Question  Response  Accuracy  -  This  was  the  measure  of  correctness  of  the  response 
to  the  situational  awareness  probe  question.  Response  accuracy  was  considered  to  be  a 
measure  of  situational  awareness.  The  subject  was  forced  to  choose  one  of  the  two 
answer  choices  provided.  A  “pass”  option  was  not  provided. 

Probe  Question  Response  Time  -  This  was  the  time  interval  from  the  time  the  electronic 
taxi  chart  appeared  until  the  question  response  button  on  the  computer  mouse  was 
depressed.  Response  time  was  considered  to  be  a  measure  of  ease  of  use  of  the  electronic 

taxi  chart. 

Pilot  Subjective  Opinion  -  Each  pilot's  subjective  opinion  was  measured  with  a  written 
questionnaire.  A  copy  of  the  questionnaire  is  available  in  Appendix.  The  completed 
questionnaires  provided  data  on  pilot  subjective  opinions  of  the  electronic  taxi  chart  and 
the  uncertainty  circles,  as  well  as  data  on  their  flight  experience. 

Test  Matrix 

Subject  pilots  were  asked  a  total  of  135  situational  awareness  probe  questions  which  were 
distributed  about  the  test  matrix  shown  in  Figure  4.9.  For  each  of  the  nine  cells  in  the  test 
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matrix,  a  version  of  each  of  the  15  situational  awareness  probe  questions  was  asked.  The 
four  versions  mentioned  in  Section  4.2.1  were  used  for  the  four  position  error  levels.  The 
same  version  was  used  for  the  constant  radius  uncertainty  circle  and  the  variable  radius 
uncertainty  circle  at  each  position  error  level.  The  15  probe  questions  were  also  asked  for 
the  no  aircraft  position  case  which  was  considered  the  baseline  case.  Only  one  probe 
question  version  was  used  for  the  “no  aircraft  position”  case  since  a  range  of  position 
error  levels  was  not  needed. 


No  Aircraft  Position 


15  Questions 


Position  Uncertainty  Symbology 

Constant  Radius 
Uncertainty  Circle 

Variable  Radius 
Uncertainty  Circle 

4.5  m 

15  Questions 

1 5  Questions 

Position 

22.5  m 

15  Questions 

1 5  Questions 

Error 

45  m 

15  Questions 

1 5  Questions 

Levels 

90  m 

15  Questions 

15  Questions 

Figure  4.9  Experimental  Test  Matrix.  Position  Error 
values  given  in  meters. 


Counterbalancing 

As  shown  in  the  counterbalancing  diagram  in  Figure  4.10,  half  of  the  subjects  received 
the  “no  aircraft  position”  questions  at  the  beginning  of  the  experiment  while  the  rest 
received  them  at  the  end.  Half  of  the  subjects  received  the  variable  radius  uncertainty 
circle  questions  before  the  constant  radius  uncertainty  circle  questions  while  the  rest 
received  them  after  constant  radius  uncertainty  circle  questions.  The  smaller  “no  aircraft 
position”  blocks  contained  15  probe  questions  while  the  larger  uncertainty  circle  blocks 
contained  60  probe  questions  spanning  the  four  position  error  levels.  All  questions  were 
asked  in  a  random  order. 
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3  Subjects 


3  Subjects 


3  Subjects 


3  Subjects 


Figure  4.10  Block  Diagram  of  Experimental  Test 
Matrix  Counterbalancing. 
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5. 


EXPERIMENTAL  PROTOCOL 


Upon  entering  the  lab,  the  subjects  were  asked  to  complete  the  first  section  of  the  written 
questionnaire  consisting  of  requests  for  each  subject's  flying  background  and  personal 
information,  as  well  as  an  informed  consent  statement.  The  remainder  of  the 
questionnaire  was  completed  after  the  experiment. 

After  filling  out  the  first  part  of  the  survey,  the  subjects  were  instructed  to  view  a  display 
of  the  aircraft  symbology  used  in  the  experiment.  This  was  done  to  familiarize  the  pilots 
with  the  meaning  of  the  symbology.  The  text  identifying  the  aircraft  symbology  was  also 
used  as  a  vision  test  to  assure  that  all  pilots  could  clearly  see  the  necessary  information  on 
the  electronic  displays.  To  assure  the  pilot  would  be  able  to  read  all  the  text  on  the 
electronic  taxi  chart,  the  text  size  on  the  vision  test  was  the  same  as  the  smallest  text  on 
the  electronic  taxi  chart. 

After  the  pilots  felt  comfortable  with  the  meaning  of  the  ownship  aircraft  symbology  they 
were  told  how  the  experiment  was  to  be  conducted.  They  were  advised  there  would  be 
four  break  periods  during  the  experiment.  They  were  also  advised  that  the  mouse  would 
be  used  to  answer  and  select  questions  throughout  the  experiment,  and  that  the  first 
priority  was  to  answer  the  questions  correctly  and  the  second  priority  was  to  answer  the 
questions  as  quickly  as  possible. 

When  they  felt  comfortable  with  the  instructions,  a  demonstration  run  was  conducted. 

The  demonstration  run  was  conducted  in  order  to  familiarize  the  pilots  with  the 
experimental  setup,  consisting  of  a  series  of  situational  awareness  probe  questions  using 
each  level  of  both  independent  variables.  After  the  demonstration  was  completed,  the 
subjects  were  asked  if  they  had  any  questions  about  the  experiment.  All  subjects 
indicated  that  they  were  comfortable  with  the  experimental  protocol  after  the 
demonstration  run  was  completed. 

Once  the  experiment  was  started,  the  subject  was  the  only  human  input  during  the 
experiment  and  the  data  was  recorded  automatically  to  avoid  experimenter  bias.  The  test 
conductor  merely  observed  the  experiment.  After  the  experiment  was  over  the  subject 
pilots  were  asked  to  fill  out  the  remaining  portion  of  the  pilot  questionnaire. 
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6.  DATA  ANALYSES 


This  chapter  explains  the  methods  that  were  used  to  analyze  data  collected  from  the 
airport  surface  situational  awareness  probe  question  experiment  and  from  the  pilot 
questionnaire  collected  at  the  time  of  the  experiment.  The  method  of  Analysis  of 
Variance  was  used  to  determine  the  effects  of  the  independent  variables  on  each  of  the 
performance  measures.  In  addition,  pairwise  comparison  tests  were  performed. 

6.1  OBJECTIVE  DATA 


A  four-way  Analysis  of  Variance  was  performed  to  determine  the  effects  of: 

•  Position  Error  (4.5,  22.5,  45,  and  90  meters) 

•  Position  Uncertainty  Symbol  Circle  Type  (Variable  or  Constant) 

•  Presentation  Order  of  Position  Icon 

no  position  icon  question  series 
position  icon  question  series 

•  Presentation  Order  of  Uncertainty  Symbol  Circle  Type 

variable  radius  circle  type  question  series 
constant  radius  circle  type  question  series 

on  each  of  the  performance  measures.  Table  6.1  shows  the  counterbalancing.  An  arcsine 
transformation  of  the  square  root  of  the  response  accuracy  percentage  was  used  to 
minimize  residual  width  variations.  Statistical  data  will  be  presented  as  both  an  F-value 
and  a  p-value. 


Table  6.1  Experimental  Counterbalancing. 


Electronic  taxi  chart  with 
no  position  icon  seen  first. 

Electronic  taxi  chart  with 
no  position  icon  seen  last 

Variable  circle  type  seen 
before  constant  circle  type 

3  subjects 

3  subjects 

Constant  circle  type  seen 
before  variable  circle  type 

3  subjects 

3  subjects  | 
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7.  RESULTS  AND  DISCUSSION 


Twelve  active  airline  pilots  were  randomly  solicited  from  a  list  of  airline  transport  pilots 
residing  in  New  England.  Their  average  age  was  39  with  a  low  of  24  and  a  high  of  50. 
The  average  flight  experience  was  9,058  hours  with  a  low  of  2,200  and  a  high  of  20,000. 
Two  of  the  pilots  flew  turboprop  aircraft  with  commuter  airlines;  the  rest  flew  turbofan 
aircraft  for  major  carriers.  Ten  of  the  twelve  pilots  had  experience  with  EFTS  aircraft.* 
Half  of  the  subjects  were  captains  and  the  other  half  were  first  officers. 

7.1  OBJECTIVE  RESULTS 

Figure  7.1  shows  the  response  accuracy  results  for  all  conditions  (mean  across  all 
subjects).  The  horizontal  dashed  line  at  approximately  78%  accuracy  is  the  mean  of  the 
results  obtained  for  those  question  where  the  pilots  were  not  given  a  position  symbol  on 


Position  Uncertainty 


Figure  7.1  Plot  of  Probe  Question  Response  Accuracy 
vs.  Position  Error. 


EFIS  aircraft  include  B737-300  and  above,  B747-400,  B757/767,  MD-80  and  above,  amd  MD-1 1 . 
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the  electronic  taxi  chart.  The  plot  shows  that  there  is  an  improvement  in  response 
accuracy  as  position  error  decreases,  and  apparently  no  advantage  of  the  type  of  circle. 
ANOVA  results  on  response  accuracy  yielded  a  significant  main  effect  for  position  error 
(F(3,24)  =  23.2,  p  <  0.001).  No  other  statistically  significant  results  were  found  (i.e.  p  < 
0.05).  Pairwise  comparisons  between  the  means  of  the  different  position  accuracies 
yielded  no  significant  differences  between  the  no  position  circle  and  the  90  meter  position 
error  conditions. 

Statistically,  significant  differences  were  found  between  the  90  and  45  meter  position 
error  conditions  (F(l  ,8)  =  22.7,  p  <  0.001),  and  between  the  45  and  22.5  meter  position 
error  conditions  (F(l,8)  =  24.0,  p  <  0.001).  There  was  no  significant  changes  in  response 
accuracy  to  probe  questions  for  position  error  less  than  22.5  meters.  These  results  suggest 
that  up  to  a  point,  an  improved  GPS  position  solution  does  enhance  situational  awareness. 

A  plot  of  these  data  is  shown  in  Figure  7.2.  Response  latency  is  plotted  as  a  function  of 
position  error  with  circle  type  as  the  parameter.  On  average,  as  position  error  decreases, 
the  response  latency  also  decreases.  This  effect  is  greater  for  the  variable  circle  condition 
than  for  the  constant  circle  condition.  A  four-way  ANOVA  yielded  several  significant 
results.  First,  there  was  a  significant  two-way  interaction  between  circle  type  and  position 
error  (F(3,22)  =  8.18,  p  <  0.001),  and  a  significant  main  effect  for  both  circle  type 
(F(3,22)  =  6.52,  p  <  0.05),  and  position  error  (F(3,22)  =  13.93,  p  <  0.001). 


Position  Uncertainty 


Figure  7.2  Plot  of  Situational  Awareness  Probe 
Question  Response  Time  vs.  GPS  Position  Error. 
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Pairwise  comparisons  between  means  indicated  significant  differences  between  constant 
and  variable  circle  conditions  at  the  90  meter  (F(l,22)  =  15.30,  p  <  0.001),  the  45  meter 
(F(l,22)  =  43.35,  p  <  0.001),  the  22.5  meter  (F(l,22)  =  54.48,  p  <  0.001),  and  the  4.5 
meter  (F(l,22)  =  1 17.74,  p  <  0.001)  position  error  levels.  Significant  differences  in 
pairwise  comparisons  were  also  found  between:  the  no  position  circle  and  the  90  meter 
variable  uncertainty  circle  conditions  (F(l,22)  =  7.45,  p  <  0.01),  the  45  meter  variable 
circle  and  the  22.5  meter  variable  circle  conditions  (F(l,22)  =  33.28,  p  <  0.001),  the  22.5 
meter  variable  circle  and  the  4.5  meter  variable  circle  conditions  (F(l,22)  =  6.65,  p  < 
0.05),  and  between  the  45  and  22.5  meter  constant  circle  conditions  (F(l,22)  =  24.73,  p  < 
0.001).  These  results  indicate  that  response  latency  is  shorter  for  all  levels  of  position 
error  for  the  variable  circle  condition  than  for  the  constant  circle  condition. 

7.2  SUB JECTIVE  RESULTS 

Each  subject  completed  a  written  questionnaire  depicting  their  opinions  on  the  electronic 
taxi  chart.  The  results  are  presented  below. 

Usefulness  of  Flight  Deck  Electronic  Taxi  Chart  Presentation 

Subjects  were  asked  to  rate  (1  to  5  scale)  the  usefulness  of  an  electronic  taxi  chart  in 
terms  of  their  day-to-day  flight  operations.  The  twelve  pilots  gave  an  average  rating  of  4 
(standard  deviation  of  1.13),  indicating  that  they  felt  the  electronic  taxi  chart  would  be 
useful  in  their  day-to-day  operations. 

The  Best  Features  of  the  Electronic  Taxi  Chart 

In  an  effort  to  identify  electronic  taxi  chart  feature  preferences,  pilots  were  asked  to 
identify  the  best  features  of  the  electronic  taxi  charts  used  in  the  experiment.  The 
subjects’  preferences  were  categorized  and  the  results  shown  in  Table  7.1.  In 
categorizing  the  results,  both  aircraft  location,  and  the  position  uncertainty  circles  were 
considered  as  aircraft  position  information.  As  shown  in  the  table,  pilots  were  most 
enthusiastic  about  having  aircraft  position  information  displayed  on  the  chart. 


Table  7.1  Subjective  Results  Showing  Most  Liked  Features  of 
Electronic  Taxi  Chart. 


Feature 

Number  of  Pilots 

Percent  of  Total 

Aircraft  Position  Information 

8 

67% 

Aircraft  Heading  Information 

5 

42% 

Airport  Surface  Features 

6 

50% 

39 


Percentage  of  Time  the  Electronic  Taxi  Chart  and  Supporting  Simulation  Were  Used 
During  the  Experiment 

In  order  to  obtain  a  measure  of  how  frequently  the  subjects  used  the  electronic  taxi  chart, 
the  subjects  were  asked  to  give  the  percentage  of  time  they  used  the  electronic  taxi  chart, 
and  the  supporting  out-the-window  view  and  EHSI  throughout  the  experiment.  The  data 
is  shown  in  a  pie  chart  in  Figure  7.3.  As  expected,  the  electronic  taxi  chart  was  used  the 
most  (51%  of  the  time),  indicating  that  pilots  considered  the  electronic  taxi  chart  the 
primary  means  for  determining  situational  awareness.  The  out-the-window  view  and 
EHSI  may  have  been  used  as  secondary  situational  awareness  tools. 


EHSI 

10% 


Out-the- 
Window 
View 
3  9% 


Electronic 
Taxi  Chart 
51% 


Figure  7.3  Plot  of  Percent  of  Time  Electronic  Taxi 
Chart  and  Supporting  Simulation  Were  Used  During 
the  Experiment. 

Rating  of  Uncertainty  Circle  by  Size  and  Position  Uncertainty  Circle  by  Type 

Seven  head-to-head  comparisons  were  made  of  the  different  position  uncertainty  circle 
conditions.  Figure  7.4  graphically  presents  the  results.  There  was  a  general  preference 
for  higher  accuracy  depiction. 
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8.  SUMMARY  AND  CONCLUSIONS 


The  advent  of  the  Global  Positioning  System  (GPS)  has  provided  a  means  of  providing 
precise  aircraft  location  information.  This  position  information,  coupled  with  current 
advanced  display  capabilities,  creates  a  cockpit-based  ground  navigation  system  which 
may  be  used  by  flight  crews  in  low-visibility  conditions  to  maintain  airport  surface 
situational  awareness,  which  is  a  measure  of  a  flight  crews’  awareness  of  their  location 
with  respect  to  airport  surface  features  such  as  runways  and  taxiways. 

This  experiment  was  designed  to  determine  the  potential  benefit  of  displaying  aircraft 
position  as  well  as  to  provide  insight  on  what  level  of  position  accuracy  may  be  needed  to 
maintain  airport  surface  situational  awareness.  In  addition,  two  types  of  position 
confidence  symbologies  were  evaluated:  the  constant  radius  uncertainty  circle  and  the 
variable  radius  uncertainty  circle. 

Situational  awareness  was  assessed  by  asking  1 2  airline  pilots  a  series  of  probe  questions 
about  their  location  on  the  airport  surface.  The  pilots  used  static  “snapshot”  images  of  a 
north-up  electronic  taxi  chart  as  well  as  a  supporting  out-the-window  view  and  aircraft 
heading  display  to  answer  the  situational  awareness  probe  questions. 

In  summary,  the  major  conclusions  of  this  study  are  the  following: 

•  Results  from  this  study  indicate  that  GPS-derived  aircraft  position  information, 
when  displayed  on  an  electronic  taxi  chart,  appears  to  enhance  situational 
awareness.  The  results  show  that  there  was  a  significant  improvement  for 
system  accuracy  better  than  50  meters. 

•  Pilots  responded  faster  to  the  situational  awareness  probe  questions  as  the 
position  error  level  was  decreased  with  the  variable  radius  uncertainty  circle.  In 
all  cases  the  subjects  responded  faster  when  presented  with  a  variable  error 
uncertainty  symbol.  This  is  thought  to  be  due  to  a  perceived  increase  in  system 
performance,  a  result  that  is  supported  by  the  response  latency  results  for  the  90 
meter  position  error  case.  Here,  both  the  variable  and  constant  radius 
uncertainty  symbols  were  identical  yet  the  pilots  showed  increased  performance 
with  the  variable  radius  uncertainty  circle  symbol. 

•  Subjective  results  supported  the  behavioral  data.  The  pilots  showed  a  stronger 
preference  for  the  variable  radius  uncertainty  circle  symbol  with  smaller  position 
uncertainty. 

•  The  aircraft  position  and  heading  symbology  used  in  this  experiment  were  well 
received  by  the  subject  pilots.  When  asked  to  identify  the  best  features  of  the 
electronic  taxi  charts,  eight  of  the  pilots  mentioned  aircraft  position  information 
and  six  mentioned  the  graphical  heading  indicator. 
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APPENDIX  -  PILOT  SUBJECTIVE  QUESTIONNAIRE 


Information  concerning  your  aviation  background  will  help  us  to  more  accurately 
assess  some  of  the  variables  that  affect  your  preferences  for  airport  surface  displays.  All 
information  you  provide  will  remain  completely  anonymous. 


1. 

2. 

3. 


Age: _  Sex:  Male  (  ) 

How  were  you  initially  trained  to  fly?  Civil 

Experience: 

A.  Total  pilot  hours _ 

B.  Pilot  ratings  held: 

Fixed  Wing:  ATP  (  )  Commercial  Pilot  ( 

Rotary  Wing:  ATP  (  )  Commercial  Pilot  ( 


Female  (  ) 

(  )  Military  (  ) 


)  F.E.  Written  (  ) 
)  Other _ 


C.  Current  flight  deck  position  (circle  one): 

Captain  or  First  Officer 

D.  Current  aircraft  type  _ 

Number  of  hours  in  this  type  _ 

E.  Please  list  other  aircraft  flown  for  significant  periods. 
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4. 


4.  When  conducting  flight  operations  in  very  low  visibility  conditions  (less  than  600’ 

RVR),  please  rate  the  difficulty  of  each  phase  of  flight  in  terms  of  maintaining 
situational  awareness. 

Not  Difficult 

Moderately  Difficult 

Very  Difficult 

Ground  Taxi 

1 

2 

3 

4 

5 

Takeoff 

1 

2 

3 

4 

5 

Climb 

1 

2 

3 

4 

5 

Cruise 

1 

2 

3 

4 

5 

Approach 

1 

2 

3 

4 

5 

Landing 

1 

2 

3 

4 

5 

5.  What  percentage  of  time  would  you  estimate  that  you  have 

conditions  of: 

taxied  in  visibility 

0-5%  5-10% 

10-20%  >20% 

0  -  600  ft.  RVR 

[  ]  [ 

] 

[  ]  [  ] 

600  -  1200  ft.  RVR 

[  ]  [ 

] 

[  ]  [  ] 

PLEASE  DO  NOT  PROCEED  FURTHER 
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Post  Experiment  Questionnaire 


1 .  What  are  the  best  features  of  the  electronic  taxi  charts  used  in  this  experiment? 


2.  What  are  the  worst  features  of  the  electronic  taxi  charts  used  in  this  experiment? 


3.  The  definition  of  a  runway  incursion  is  as  follows: 

Any  occurrence  at  an  airport  involving  an  aircraft,  vehicle,  person,  or 
object  on  the  ground  that  creates  a  collision  hazard  or  results  in  loss  of 
separation  with  an  aircraft  taking  off,  intending  to  take  off,  landing,  or 
intending  to  land. 

A.  Have  you  ever  been  involved  in  a  runway  or  taxiway  incursion?  Please 
describe  it  or  the  ‘closest  call’. 
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B.  How  could  this  incident  have  been  prevented? 


C.  Could  the  airport  surface  taxi  charts  with  ownship  position  used  in 
this  experiment  help  to  aid  in  the  prevention  of  this  incident? 


4.  In  terms  of  day-to-day  operations,  how  useful  would  a  flight  deck  electronic  taxi 
chart  with  ownship  position  display  be? 

1  2  3  4  5 

Not  Useful  Useful  Very  Useful 

5.  Did  you  feel  the  size  of  the  error  circle  had  an  effect  on  the  time  it  took  you  to 
locate  the  ownship  icon  on  the  taxi  display?  (Please  circle  one) 

1  2  3  4  5 

No  effect  Moderate  Effect  Large  Effect 

6.  If  a  variable  error  circle  ownship  icon  was  employed,  what  is  the  maximum  size 
error  circle  you  would  feel  comfortable  taxiing  with?  (please  circle  one) 

5M  25M  50M  100M 


7. 


Please  list  the  percentage  of  time  you  used  each  of  the  three  displays  throughout 
the  experiment. 

Electronic  Taxi  Chart _ 


Out-the-Window  View 


EHSI 
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5M  RADIUS  ERROR  CIRCLE  ICON  vs.  100M  RADIUS  ERROR  CIRCLE  ICON 
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25M  RADIUS  ERROR  CIRCLE  ICON  vs.  50M  RADIUS  ERROR  CIRCLE  ICON 
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